Towards Outcome-Driven Patient Subgroups: A Machine Learning Analysis Across Six Depression Treatment Studies.

BACKGROUND: Major depressive disorder (MDD) is a heterogeneous condition; multiple underlying neurobiological and behavioral substrates are associated with treatment response variability. Understanding the sources of this variability and predicting outcomes has been elusive. Machine learning (ML) shows promise in predicting treatment response in MDD, but its application is limited by challenges to the clinical interpretability of ML models, and clinicians often lack confidence in model results. In order to improve the interpretability of ML models in clinical practice, our goal was to demonstrate the derivation of treatment-relevant patient profiles comprised of clinical and demographic information using a novel ML approach. METHODS: We analyzed data from six clinical trials of pharmacological treatment for depression (total n = 5438) using the Differential Prototypes Neural Network (DPNN), a ML model that derives patient prototypes which can be used to derive treatment-relevant patient clusters while learning to generate probabilities for differential treatment response. A model classifying remission and outputting individual remission probabilities for five first-line monotherapies and three combination treatments was trained using clinical and demographic data. Prototypes were evaluated for interpretability by assessing differences in feature distributions (e.g. age, sex, symptom severity) and treatment-specific outcomes. RESULTS: A 3-prototype model achieved an area under the receiver operating curve of 0.66 and an expected absolute improvement in remission rate for those receiving the best predicted treatment of 6.5% (relative improvement of 15.6%) compared to the population remission rate. We identified three treatment-relevant patient clusters. Cluster A patients tended to be younger, to have increased levels of fatigue, and more severe symptoms. Cluster B patients tended to be older, female, have less severe symptoms, and the highest remission rates. Cluster C patients had more severe symptoms, lower remission rates, more psychomotor agitation, more intense suicidal ideation, and more somatic genital symptoms. CONCLUSION: It is possible to produce novel treatment-relevant patient profiles using ML models; doing so may improve interpretability of ML models and the quality of precision medicine treatments for MDD.

A matching procedure for sequential experiments that iteratively learns which covariates improve power.

We propose a dynamic allocation procedure that increases power and efficiency when measuring an average treatment effect in sequential randomized trials exploiting some subjects' previous assessed responses. Subjects arrive sequentially and are either randomized or paired to a previously randomized subject and administered the alternate treatment. The pairing is made via a dynamic matching criterion that iteratively learns which specific covariates are important to the response. We develop estimators for the average treatment effect as well as an exact test. We illustrate our method's increase in efficiency and power over other allocation procedures in both simulated scenarios and a clinical trial dataset. An R package "SeqExpMatch" for use by practitioners is available on CRAN.

Do book consumers discriminate against Black, female, or young authors?

The publishing industry shows marked evidence of both gender and racial discrimination. A rational explanation for this difference in treatment of both female and Black authors might relate to the taste-based preferences of book consumers, who might be less willing to pay for books by such authors. We ran a randomized experiment to test for the presence of discriminatory preferences by consumers based on authors' race, gender and/or age. We collected ratings of 25,201 book surveys across 9,072 subjects on Amazon's Mechanical Turk, making this study the largest experimental study of the book market to date. Subjects were presented with mocked-up book covers and descriptions from each of 14 fiction and non-fiction genres, with one of three possible titles per book randomly assigned. Using author names and photographs, we signaled authors' race, gender, and age and randomly assigned these combinations to each book presented to our subjects. We then asked subjects to rate their interest in purchasing the book, their evaluation of the author's credentials, and the amount they were willing to pay for the book. The experimental design of this study strived to eliminate the potential for proxy-based discrimination by providing book descriptions that detailed the authors' relevant professional experience. The large sample allowed for exploration of various types of taste-based discrimination observed in the literature, including discrimination against particular groups, homophily, and pro-social behavior. Overall, book consumers showed a willingness to pay approximately $0.50 or 3.5% more on average for books by Black authors and little, if any, practically meaningful discrimination based on age or gender. In other words, our study finds no and even contrary evidence of taste-based preferences by consumers that would rationalize the historic discriminatory treatment of Black or of female authors by publishers nor of discrimination based on an author's age.

Treatment selection using prototyping in latent-space with application to depression treatment.

Machine-assisted treatment selection commonly follows one of two paradigms: a fully personalized paradigm which ignores any possible clustering of patients; or a sub-grouping paradigm which ignores personal differences within the identified groups. While both paradigms have shown promising results, each of them suffers from important limitations. In this article, we propose a novel deep learning-based treatment selection approach that is shown to strike a balance between the two paradigms using latent-space prototyping. Our approach is specifically tailored for domains in which effective prototypes and sub-groups of patients are assumed to exist, but groupings relevant to the training objective are not observable in the non-latent space. In an extensive evaluation, using both synthetic and Major Depressive Disorder (MDD) real-world clinical data describing 4754 MDD patients from clinical trials for depression treatment, we show that our approach favorably compares with state-of-the-art approaches. Specifically, the model produced an 8% absolute and 23% relative improvement over random treatment allocation. This is potentially clinically significant, given the large number of patients with MDD. Therefore, the model can bring about a much desired leap forward in the way depression is treated today.

Evaluating the Effectiveness of Personalized Medicine With Software.

We present methodological advances in understanding the effectiveness of personalized medicine models and supply easy-to-use open-source software. Personalized medicine involves the systematic use of individual patient characteristics to determine which treatment option is most likely to result in a better average outcome for the patient. Why is personalized medicine not done more in practice? One of many reasons is because practitioners do not have any easy way to holistically evaluate whether their personalization procedure does better than the standard of care, termed improvement. Our software, "Personalized Treatment Evaluator" (the R package PTE), provides inference for improvement out-of-sample in many clinical scenarios. We also extend current methodology by allowing evaluation of improvement in the case where the endpoint is binary or survival. In the software, the practitioner inputs 1) data from a single-stage randomized trial with one continuous, incidence or survival endpoint and 2) an educated guess of a functional form of a model for the endpoint constructed from domain knowledge. The bootstrap is then employed on data unseen during model fitting to provide confidence intervals for the improvement for the average future patient (assuming future patients are similar to the patients in the trial). One may also test against a null scenario where the hypothesized personalization are not more useful than a standard of care. We demonstrate our method's promise on simulated data as well as on data from a randomized comparative trial investigating two treatments for depression.

The Bayesian Additive Regression Trees Formula for Safe Machine Learning-Based Intraocular Lens Predictions.

Purpose: Our work introduces a highly accurate, safe, and sufficiently explicable machine-learning (artificial intelligence) model of intraocular lens power (IOL) translating into better post-surgical outcomes for patients with cataracts. We also demonstrate its improved predictive accuracy over previous formulas. Methods: We collected retrospective eye measurement data on 5,331 eyes from 3,276 patients across multiple centers who received a lens implantation during cataract surgery. The dependent measure is the post-operative manifest spherical equivalent error from intended and the independent variables are the patient- and eye-specific characteristics. This dataset was split so that one subset was for formula construction and the other for validating our new formula. Data excluded fellow eyes, so as not to confound the prediction with bilateral eyes. Results: Our formula is three times more precise than reported studies with a median absolute IOL error of 0.204 diopters (D). When converted to absolute predictive refraction errors on the cornea, the median error is 0.137 D which is close to the IOL manufacturer tolerance. These estimates are validated out-of-sample and thus are expected to reflect the future performance of our prediction formula, especially since our data were collected from a wide variety of patients, clinics, and manufacturers. Conclusion: The increased precision of IOL power calculations has the potential to optimize patient positive refractive outcomes. Our model also provides uncertainty plots that can be used in tandem with the clinician's expertise and previous formula output, further enhancing the safety. Translational relavance: Our new machine learning process has the potential to significantly improve patient IOL refractive outcomes safely.

Accelerating Adolescent Vocabulary Growth: Development of an Individualized, Web-Based, Vocabulary Instruction Program.

Purpose Improving vocabulary knowledge is important for many adolescents, but there are few evidence-based vocabulary instruction programs available for high school students. The purpose of this article is to describe the iterative development of the DictionarySquared research platform, a web-based vocabulary program that provides individualized vocabulary instruction, and to report the results of 2 pilot studies examining the feasibility of implementation and potential effectiveness with high school students. Method We describe our theory of change and 5 phases of iterative development. In Phases 1 and 2, we evaluated the initial implementation of DictionarySquared and revised the instructional materials. In Phase 3, we conducted a feasibility study involving 169 high school students who used the program for 4 weeks. Student usage data and feedback from teachers were used to guide program revisions in Phase 4. In Phase 5, we examined potential effectiveness for 264 high school students who were assigned to use the program for 1 semester. Results Results of the Phase 3 study indicated that implementation of the program was feasible, although usage was below assigned levels. Results of the Phase 5 study indicated that the duration of active program usage significantly predicted posttest vocabulary scores on the proximal assessment after controlling for pretest standardized vocabulary scores. Analyses using propensity score matching revealed positive, but nonsignificant, gains on standardized assessments between pre- and posttests. Conclusion Together, the results of early-stage pilot studies are promising and suggest that a more rigorous test of efficacy is warranted. Successful implementation of the DictionarySquared research program, as well as lessons learned from the program development process, will expand the range of evidence-based treatment options that clinicians and educators can use to improve adolescent vocabulary and reading comprehension skills. Supplemental Material https://doi.org/10.23641/asha.9765161.

Comparing gender discrimination and inequality in indie and traditional publishing.

In traditional publishing, female authors' titles command nearly half (45%) the price of male authors' and are underrepresented in more prestigious genres, and books are published by publishing houses, which determined whose books get published, subject classification, and retail price. In the last decade, the growth of digital technologies and sales platforms have enabled unprecedented numbers of authors to bypass publishers to publish and sell books. The rise of indie publishing (aka self-publishing) reflects the growth of the "gig" economy, where the influence of firms has diminished and workers are exposed more directly to external markets. Encompassing the traditional and the gig economy, the book industry illuminates how the gig economy may disrupt, replicate, or transform the gender discrimination mechanisms and inequality found in the traditional economy. In a natural experiment spanning from 2002 to 2012 and including over two million book titles, we compare discrimination mechanisms and inequality in indie and traditional publishing. We find that indie publishing, though more egalitarian, largely replicates traditional publishing's gender discrimination patterns, showing an unequal distribution of male and female authors by genre (allocative discrimination), devaluation of genres written predominantly by female authors (valuative discrimination), and lower prices within genres for books by female authors (within-job discrimination). However, these discrimination mechanisms are associated with far less price inequality in indie, only 7%, in large part due to the smaller and lower range of prices in indie publishing compared to traditional publishing. We conclude that, with greater freedom, workers in the gig economy may be inclined to greater equality but will largely replicate existing labor market segmentation and the lower valuation of female-typical work and of female workers. Nonetheless, price setting for work may be more similar for workers in the gig economy due to market competition that will compress prices ranges.

An examination of biochemical parameters and their association with response to ketogenic dietary therapies.

OBJECTIVE: In the absence of specific metabolic disorders, accurate predictors of response to ketogenic dietary therapies (KDTs) for treating epilepsy are largely unknown. We hypothesized that specific biochemical parameters would be associated with the effectiveness of KDT in humans with epilepsy. The parameters tested were ß-hydroxybutyrate, acetoacetate, nonesterified fatty acids, free and acylcarnitine profile, glucose, and glucose-ketone index (GKI). METHODS: Biochemical results from routine blood tests conducted at baseline prior to initiation of KDT and at 3-month follow-up were obtained from 13 adults and 215 children with KDT response data from participating centers. One hundred thirty-two (57%) of 228 participants had some data at both baseline and 3 months; 52 (23%) of 228 had data only at baseline; 22 (10%) of 228 had data only at 3 months; and 22 (10%) of 228 had no data. KDT response was defined as ≥50% seizure reduction at 3-month follow-up. RESULTS: Acetyl carnitine at baseline was significantly higher in responders (p < 0.007). It was not associated with response at 3-month follow-up. There was a trend for higher levels of free carnitine and other acylcarnitine esters at baseline and at 3-month follow-up in KDT responders. There was also a trend for greater differences in levels of propionyl carnitine and in ß-hydroxybutyrate measured at baseline and 3-month follow-up in KDT responders. No other biochemical parameters were associated with response at any time point. SIGNIFICANCE: Our finding that certain carnitine fractions, in particular baseline acetyl carnitine, are positively associated with greater efficacy of KDT is consistent with the theory that alterations in energy metabolism may play a role in the mechanisms of action of KDT.

Optimal experimental designs for estimating Henry's law constants via the method of phase ratio variation.

When measuring Henry's law constants (kH) using the phase ratio variation (PRV) method via headspace gas chromatography (GC), the value of kH of the compound under investigation is calculated from the ratio of the slope to the intercept of a linear regression of the inverse GC response versus the ratio of gas to liquid volumes of a series of vials drawn from the same parent solution. Thus, an experimenter collects measurements consisting of the independent variable (the gas/liquid volume ratio) and dependent variable (the GC-1 peak area). A review of the literature found that the common design is a simple uniform spacing of liquid volumes. We present an optimal experimental design which estimates kH with minimum error and provides multiple means for building confidence intervals for such estimates. We illustrate performance improvements of our design with an example measuring the kH for Naphthalene in aqueous solution as well as simulations on previous studies. Our designs are most applicable after a trial run defines the linear GC response and the linear phase ratio to the GC-1 region (where the PRV method is suitable) after which a practitioner can collect measurements in bulk. The designs can be easily computed using our open source software optDesignSlopeInt, an R package on CRAN.

PREDICTING INDIVIDUAL WELL-BEING THROUGH THE LANGUAGE OF SOCIAL MEDIA.

We present the task of predicting individual well-being, as measured by a life satisfaction scale, through the language people use on social media. Well-being, which encompasses much more than emotion and mood, is linked with good mental and physical health. The ability to quickly and accurately assess it can supplement multi-million dollar national surveys as well as promote whole body health. Through crowd-sourced ratings of tweets and Facebook status updates, we create message-level predictive models for multiple components of well-being. However, well-being is ultimately attributed to people, so we perform an additional evaluation at the user-level, finding that a multi-level cascaded model, using both message-level predictions and userlevel features, performs best and outperforms popular lexicon-based happiness models. Finally, we suggest that analyses of language go beyond prediction by identifying the language that characterizes well-being.

Starvation of cancer via induced ketogenesis and severe hypoglycemia.

Neoplasms are highly dependent on glucose as their substrate for energy production and are generally not able to catabolize other fuel sources such as ketones and fatty acids. Thus, removing access to glucose has the potential to starve cancer cells and induce apoptosis. Unfortunately, other body tissues are also dependent on glucose for energy under normal conditions. However, in human starvation (or in the setting of diet-induced ketogenesis), the body "keto-adapts" and glucose requirements of most tissues drop to almost nil. Exceptions include the central nervous system (CNS) and various other tissues which have a small but obligatory requirement of glucose. Our hypothesized treatment takes keto-adaptation as a prerequisite. We then propose the induction of severe hypoglycemia by depressing gluconeogenesis while administering glucose to the brain. Although severe hypoglycemia normally produces adverse effects such as seizure and coma, it is relatively safe following keto-adaptation. We hypothesize that our therapeutic hypoglycemia treatment has potential to rapidly induce tumor cell necrosis.

Matching on-the-fly: sequential allocation with higher power and efficiency.

We propose a dynamic allocation procedure that increases power and efficiency when measuring an average treatment effect in fixed sample randomized trials with sequential allocation. Subjects arrive iteratively and are either randomized or paired via a matching criterion to a previously randomized subject and administered the alternate treatment. We develop estimators for the average treatment effect that combine information from both the matched pairs and unmatched subjects as well as an exact test. Simulations illustrate the method's higher efficiency and power over several competing allocation procedures in both simulations and in data from a clinical trial.

Spatial organization of dendritic cells within tumor draining lymph nodes impacts clinical outcome in breast cancer patients.

BACKGROUND: Dendritic cells (DCs) are important mediators of anti-tumor immune responses. We hypothesized that an in-depth analysis of dendritic cells and their spatial relationships to each other as well as to other immune cells within tumor draining lymph nodes (TDLNs) could provide a better understanding of immune function and dysregulation in cancer. METHODS: We analyzed immune cells within TDLNs from 59 breast cancer patients with at least 5 years of clinical follow-up using immunohistochemical staining with a novel quantitative image analysis system. We developed algorithms to analyze spatial distribution patterns of immune cells in cancer versus healthy intra-mammary lymph nodes (HLNs) to derive information about possible mechanisms underlying immune-dysregulation in breast cancer. We used the non-parametric Mann-Whitney test for inter-group comparisons, Wilcoxon Matched-Pairs Signed Ranks test for intra-group comparisons and log-rank (Mantel-Cox) test for Kaplan Maier analyses. RESULTS: Degree of clustering of DCs (in terms of spatial proximity of the cells to each other) was reduced in TDLNs compared to HLNs. While there were more numerous DC clusters in TDLNs compared to HLNs,DC clusters within TDLNs tended to have fewer member DCs and also consisted of fewer cells displaying the DC maturity marker CD83. The average number of T cells within a standardized radius of a clustered DC was increased compared to that of an unclustered DC, suggesting that DC clustering was associated with T cell interaction. Furthermore, the number of T cells within the radius of a clustered DC was reduced in tumor-positive TDLNs compared to HLNs. Importantly, clinical outcome analysis revealed that DC clustering in tumor-positive TDLNs correlated with the duration of disease-free survival in breast cancer patients. CONCLUSIONS: These findings are the first to describe the spatial organization of DCs within TDLNs and their association with survival outcome. In addition, we characterized specific changes in number, size, maturity, and T cell co-localization of such clusters. Strategies to enhance DC function in-vivo, including maturation and clustering, may provide additional tools for developing more efficacious DC cancer vaccines.

Quantitative, architectural analysis of immune cell subsets in tumor-draining lymph nodes from breast cancer patients and healthy lymph nodes.

BACKGROUND: To date, pathological examination of specimens remains largely qualitative. Quantitative measures of tissue spatial features are generally not captured. To gain additional mechanistic and prognostic insights, a need for quantitative architectural analysis arises in studying immune cell-cancer interactions within the tumor microenvironment and tumor-draining lymph nodes (TDLNs). METHODOLOGY/PRINCIPAL FINDINGS: We present a novel, quantitative image analysis approach incorporating 1) multi-color tissue staining, 2) high-resolution, automated whole-section imaging, 3) custom image analysis software that identifies cell types and locations, and 4) spatial statistical analysis. As a proof of concept, we applied this approach to study the architectural patterns of T and B cells within tumor-draining lymph nodes from breast cancer patients versus healthy lymph nodes. We found that the spatial grouping patterns of T and B cells differed between healthy and breast cancer lymph nodes, and this could be attributed to the lack of B cell localization in the extrafollicular region of the TDLNs. CONCLUSIONS/SIGNIFICANCE: Our integrative approach has made quantitative analysis of complex visual data possible. Our results highlight spatial alterations of immune cells within lymph nodes from breast cancer patients as an independent variable from numerical changes. This opens up new areas of investigations in research and medicine. Future application of this approach will lead to a better understanding of immune changes in the tumor microenvironment and TDLNs, and how they affect clinical outcomes.

An Interactive Java Statistical Image Segmentation System: GemIdent.

Supervised learning can be used to segment/identify regions of interest in images using both color and morphological information. A novel object identification algorithm was developed in Java to locate immune and cancer cells in images of immunohistochemically-stained lymph node tissue from a recent study published by Kohrt et al. (2005). The algorithms are also showing promise in other domains. The success of the method depends heavily on the use of color, the relative homogeneity of object appearance and on interactivity. As is often the case in segmentation, an algorithm specifically tailored to the application works better than using broader methods that work passably well on any problem. Our main innovation is the interactive feature extraction from color images. We also enable the user to improve the classification with an interactive visualization system. This is then coupled with the statistical learning algorithms and intensive feedback from the user over many classification-correction iterations, resulting in a highly accurate and user-friendly solution. The system ultimately provides the locations of every cell recognized in the entire tissue in a text file tailored to be easily imported into R (Ihaka and Gentleman 1996; R Development Core Team 2009) for further statistical analyses. This data is invaluable in the study of spatial and multidimensional relationships between cell populations and tumor structure. This system is available at http://www.GemIdent.com/ together with three demonstration videos and a manual.